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1 26.08.2002 
Method of Co»teiit Identification, Device, and Software 



The invention relates to a method of cont^ idCTiafication, comprising the step 
of creating a first signature for a first content Item comprising a first sequence of frames. 

The invention further relates to an electronic device comprising an inter&ce 
for inter&oing with a storage means storing a first signature of a first content item, the first 
S content x^tn comprising a first sequence of frames; a receiver able to recdive a signal 

comprising a second content item» the second content item comprising a second sequence of 
frames and a control unit able to use the interface to tetdeve the fii:st signature &om the 
storage means, able to ta^te a second signature for ttie second cont^t it^ and able to 
detenuine similarity betwecm the first signature and fixe second siguature. 
10 The invention fatOxex relates to sofiware enabling upon its .execution a 

programmable device to function as an electronic device. 



An embodiment of the method is known from HP 0 248 533. Thekno^ 
15 method performs teal-time continuous pattern recognition of broadcast segments by 

.cor^tructing^ digital signature from .a Imown specimen of a segmeut,..wJhichis.to.be ..... 

recognized. The signature is constructed by digitally parametiziiig the segment, selecting 
. porticms atnong random firame locations tiiioui^iout the segment in accordance wifii a set of 
predefined rules to form the signature, and associating with the $igaature tisc frame locations 
20 of the portions. The Imown method is clBimed to be able to identify large mimbers of 

oommemials in an efficient and economic manner in real time, without r esorting to exi^ensive 
parallel processii^ or to the most pow^rfiil computers. 

As a drawback of the known method> it can only be ^ecuted in real time in an 
economic maimer if the number of random frame locations is limited. Unfortunately^ limitbg 
25 the number of frame locations also limits the reliability of the pattern recognition. 
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It is a first object of the invention to provide a method of the Idtid described in 

the opening paragraph, which can be executed in real time in an economic manner while 

acMeviag a relatively high reliabiKty of pattern recognition. 

It is a second object of the invention to provide an electronic device of the 

kind described in the opening paragraph, which is able to perform real-time pattern 

recogaition with a relatively high reliability. 

It is a third object of the invention to provide software of the kind described in 

the opening paragraph, which can be Ktecated in real tune in an economic manner while 

achieving a relatively high reliability of pattern recogoitiorL 



The first object is according to the invention realized in that: the step of 
creating the first signature comprises creating a fir^t sub-^signature to comprise a £rst 
sequence of first avetage$» a first average being striokm of values of a feature in multiple 

15 ftames in the first sequence of frames. A feature may for example be frame luminance, frame 
complexity, Mean Absolute Difference (MAD) error as used by the MPEQ2 encoders:, or 
scale &ctor as used by MPEG audio encoders. A frame may be an audio frame, a video 
frame^ or a synchronized audio and video frame. 

In an embodiment of the method of die invention, further comprised id the step 

20 of creating a second signature fbr a second contmt itotn comprising a second sequence of 
frames; ihe step of c];eating the second signature copprisea creating a second sub-signature to 
comprise a second sequence of second averages, a second average being stricken of values of 
the jfeature in multiple frames in the second sequence of frames; &rther comprised is the step 
of detenmning similarity between the first and the second signature; and the fStqp of 

25 detemuning similarity between the first and the second signature comprises determining 



similarity between the first and the second sub-signature. 

Similarity between the first and the second signature may be used to identic a 
short audioMdeo sequence in other streams. For real-time comparison of tens or even 
hundreds of signatured compntadonal efforts must be low. A signature of new content may be 
30 generated and compared to a database of sigoatures every N frames, Comparing signatures 
every fitime will be too computational intensive and even unnecessarily accurate in time. The 
signatures must be robust 1^ noise and other distortbns becaxiae a Personal Video Recorder- 
like device could have many different input sources rangnig from high quality digital video 
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data to low quality aaaalogue cable or VHS signals. By averaging over multiple frames, the 
efifects of noise and other distortions ate reduced. 

In an embodiment of the method of the invention, the step of determining 
similarity between the first and the second signature comprises calculating a coefficient of 
5 correlation between the first aad the second signature and comparmg the coefficient with a 
threshold. By averaging over multiple frames, a data set with a more or less nocmal 
distribution is obtained. The normal-ness of the distribution depends on the amoimt of frames 
being averaged. A good measure of similaiity can be obtained by couelating two data sets 
with a normal distdbutioxi, ag. using Pearson's correlation. Alternatively, a first average of a 
10 sequence of feaixire values could be subtracted from a second average of a sequence of 
fbature values to obtain a different similarity measmre. By ooinparing a dmilaiity measure 
witih a threshold, a positive or negative identification can be obtained, which can be the basis 



The step of determining similarity between the frt^ and the second si^iatore 
IS may ccmq>iise calculating a ooefScl^ of correlation between a first sub-sequence at a 
position in the first sequence of averages and multiple second sub-sequences in the 
neighborhood of a corresponding position in the second sequence of overages. This reduces 
the time-sjbifiing problem, v^ere for instance a missing frame in a content item might lead to 
a negative identificatioiL Frames maybe lost when diaplaymg older VHS source material. 
20 Sometimes the vertical ^chronization is missed, resultmg in lost frames. The time-shifrmg 
problem may also occur when a signature Is not created every frame^ but every multqile..-.. ^ 
frameSi 

Tto coef&cient of correlation between the first sub-sequence and the multiple 
second sub-sequences may be calculated using wei^ts» a wei^t behig larger if a second 
25 sub-sequence is near the corresponding position and smaller if a second sub-sequence is 

remote from the corresponding position^ Since lime sfaifis between similar contot items will 
more lik6ly be minor than naajor, coirelation is more likely to be accidental if tibie second 
element is t^xtote from the corresponding position. Better idmtification can be achieved 
using wd^ts. 

30 The step of creating a signature may comprise creating multiple sub-signatures 

and similarity between the first and the second signature is determined using the multiple 
sub-signatures. Although one sub«*fiignature pi^ sigrtatqre may be sufficient m some 
instances, the combinatorial behavior of low-level AV features of a short video sequence is 
more likely to be ituique to this sequence. The uniqueness of a sigr^ature comprising multiple 
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4 26.08.2002 
sub-pignatures signature depends on the amount of infonnation it represents. The longer the 
fcatore sequence? Uib more unique the signature can be. Also, the more different types of 
features are used simultaneously, and thus the more sub-signatures, liie more unique the 
sigmture can be. Due to the uniqueness of a signature, a large number of signatures can be 
5 uniquely identified under a variety of conditions usiag a single, pre-defined, identification 
criterion. In case a service provider provides the signatures, the identification criterion could 
in principal be designed per signature. This is because the service provider is able to test 
identification criteria for a signature on a largo amount of content beforehand. However;, in 
case of signatures defined by a user a single, pre-defined, identifioalion criterion ^ould 

10 suffice for all signatures. 

Creating a snb-dgnature may comprise reducing the number of averages. This 
reduces the required amount of processing. Since feature values are average^ sub-signatures 
can be sub-sampled without losing significant infomiatlon. Large differences between valura 
are more significant than small diffetences. Since differences between avers^e feature values 

IS will be smaller than difEbrences between feature values, the amount of average feature values 
can be smaller than the amount of feature values. 

If the second content item is comprised in a third content item and the first and 
the second signature are similar, a fiuther step maybe comprised slapping the second cont^t 
item in the third content Item. "Bot instance^ a signature could be made for an intro of a 

20 commercial blocQc Whenever fiie intzo is identified 3 minutes could be skipped. 

AltOTWtiyely, ^ signature could be niade for a black or blue screen that is shown when no 
signal is present. The skipping could be done automatically or the user could press a button to 
l^p a ^ven amount of content 

A fbrfheir step may be comprised identifying boundaries between a first 

25 segment and a second segment of a third content item and another step may be comprised 
fiikipping the segment in the third content item if the second content item conxprises the 
first segmmt and file fixat and the second signature are similar. The first segment may £hr 
instance be a oommexcial. The second segm^t may for ic^tance be another comnoLercial or a 
part of a movie. The segmente of commercial blocks can be identified by the use of more 

30 general discriminators and separators in the A/V domain. Segments that are inside a 

commeicial block can be detected reliably and even die boundaries betwe^ segmaits can be 
identified* Th^ signatures of detected segments can be stored in a database. New incoming 
content canbe conelated in real-time with the existing signatures of segmeaits in the database 
and if the cotrelation is high raough the content will be tagged as commercial segment. Due 
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to the feet that segments of conimercial blocks are of repetitive nature and vary infheir 
position inside a conameroial bloclc, there is a good chance to learn reliable signatures of 
xmknown commercials. With this method, the precision of a commeroial block detector can 
be increased significantly. 
5 A fiirther st^ may be comprised recording the second content item if the first 

and the second signature are similar. If the first signature was made for an inteo of a comedy 
series, a Personal Video Recorder CPVR) using the method of the invention may start 
recording as soon as the first and the second signature are found to be similar. Recording may 
ateo be started in retroaction, uang a fime-sbifling mechanism. This is uaeM when the 
10 generic intro of a series is not at the begbaning of the program. Hie first signatnreb a 

recording start-time and end-time relative to the position of flie first sequence of ftames in the 
first content item, and a set of ohannBla to scan fiMTlhe second signature could be ^ven by the 
user or downloaded ftom a sarvice provider. The method of the nrvention may also be used to 
search for a second signature in a database, retrieve fb» accompanying second content item 
IS fiom the database, and store the second contsut item. 

A fiirfher step maybe ccsnipriaed geaierating an alert if liie first and the second 
signfltat© ace shniJar. A PVR nsing the method of tiiB inventiott may alert a user by showing 
the content of interest in a Picture In Picture (PlP) window, with an icon and/or sound. The 
user couM then decide to switch to Uie identified content by pressing a button on 1h© remote 
20 control or to remove the alert When the user switcihes to the identified content, he or she 
could start watehihg the identified content live or plav. in retroactjon. fitmi ft a h agftmit^g f^f 
file content using a time-shift mechanism. 

The sacand object is according to the incventim realized in fiutt! the control 
nnit is able to create a first sub-signature ftom fiie first signature^ tiie first sub-signature 
25 comprising a first sequfflice of averages of vahies of a feature in multiple fiames in the first 
sequence of frames; the control unit is able to create a second sub-signature fbr the second 
signature by avetaghig values of the feature in multiple ftames in the second sequence of 
ftames; the control unit is able to determine similaiity between the first and the second sub- 
signature; and the control unit is able to detecmine similarity betwerai fiie first and the second 
30 signature in dependency of the similarily between fixe first and the second sub-signature. The 
device of the iiro'eottenmay be aPeesonal Video Recorder CPVR), a digital TV, or a satellite 
receiver. The control unit may be a micnoprtjcessor. The interf^ maybe a memory bus, an 
IDE hiterfkoe, or an lEBE 1394 interface. The mtetfece may have an internal or an external 
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cottnectot. The storage means may be an internal Hard disk or an external device. The 
external devicse may be located at the site of a service provider. 

lo an embodunent of liie device of the invention, the control unit is atole to 
deteimin© similarly between flie flist and the second signature by calottlatine a coefficient of 
5 correlation between the first and the second wgnature and coinparing ihe coefficient wilh a 
threshold. 

If the second content item is oomprisedinathird content item and the first and 
ae second signature are similar, ihe control nnit may be iflile to wge a finther storage means 
to store the third content item witiiout Ifae second content item. 

The control nnit may be able to urge a flntiier storage means to store the 
second content item if the first and the second signature are similar. 

The control unit may be able to generate an alett if llie first and Hie second 

sigoature are BimUar. 

The third object is according to the nivention realized in that the software 
1 5 comprises a fimction for creating a signature &r a content item comprising a sequ«ioe of 
fiames. the fimction comprising creating a sub-signature to conqwise a sequence of averages, 
an average bemg stricken of values of a feature in multipb frames in the seqmmoe of frames. 

In an embodiment of the software of the invention, fhxfher comprised is a 
function for determining similarity between two agnatures by calculating a coefficient of 
20 correlaiionbetweenthetwosignatQresandcomparingthecoefficleiAvdthadne* 

The SQflv?are may.be stOTcd on.an information carrier, mc3x as a magnedo mfi> 
earner, e.g. a floppy disk, or an optical info carrier* e.g. a C33. 



These and other aspects of the method and device of the inveudon will be 
fiirfher elucidated and described wiUi reterenoe mn& drawings, inrwhtclK 

Fig.l is a flow diagram of a fevorable embodiment of the method; 

Hg^ is a flow diagram detailing a first and a second step of Fig.l; 

Flg,3 is a flow diagram detailing athird atq> of Fig.l; 

Fig.4 is a block diagram of an embodiment of the electronic device; 

Fig^ is a schematic lepxesentation of two steps of Eig.2; 

i?ig.6 is a sdiematic representation of a variation on the two steps of Bg.5; 

Corresponding elements within the drawings are identified by the same 
reference numaal. 



CO. Hens, ceioc: x«f««t£ 



rnxuiro uxr ni- 'rax cr<4Mo^ 



PHNL020800: 



013 26.08.2002 14:4] 



26.08.2002 



The method of Fig.l comprises a step 2 creating a first signature fer a firat 
content item comprising a first sequence of frames. Step 2 comprises creating a first sub- 
5 signature to comprise a first sequence of first averages, a first average being atriokfiai of 
values of a feature in multq)le frames in the first sequence of firames. 

The method of Fig.l may finrfher comprise a step 4 creating a second signature 
for a second content item comprising a second sequence of firames and a step 6 deteocmining 
similarily between the first and the second signature. Step 4 comprises oreotiag a second sub- 
10 signature to comprise a second sequence of second averages, a second average being stricken 
of values of the feature in multiple firames in the second sequence of frames. Step 6 
comprises determining similarity between the first and the second sob-sign^taro. 

Steps 2 and 4 may comprise creating multiple sub-sigoatuies and 8imiJ«rity 
between fiie first and ihe second fiignatore naay be determined using the multiple sdb- 
15 signatures. 

If the second content item is comprised hi atiurd content item and the first and 
the second signature are similar, an optional step 8 enables skipping the second ccntent item 
in the third content item. A finrtfaer step may be comprised identift^ boundaxies betvsrecm a 
first segment and a second segment of a third content item. Optional stap 10 enables skipping 
20 the first segment in the third content item ifflie second contem item comprises t^ 
segrassnt and the first and the second signature are similar, .pptional^.l2.en*^ 
the second contBat item if the first and the second signature are similar. Optional step 14 

eiiablea generating an alert if the first and the second signature are siinilar. 

Steps 2 and 4 shown in Fig,l maybothbe subdivided into three steps, see 

25 Pig.2. Step 22» see also Fig.5, creates a sequence fe8tiireSeq(j,k) of feature values ftom a 
feature 1^ in multiple firames of a sequence of flames, k is ^ unique identifier for fba sequence 
of firames. ContentOc) is the content item coa^iising the sequence of flames. Time(k) is the 
time instance of the last flame of the sequence of fitanes expressed as a fiame number in 
contentCfc), Feature (C, p, j) is the value of feature 1^ at time instance p in content item C. The 

30 sequence of&ature values will have letigihL. 



featureSeqC/,^;) 
[feature(contwt(A:),time(;i:) 



-L+IJ) ... featoreCcontent(A),time(fc),^] 
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Step 24, see also Fig.5, creates a first suWguataie using ifae sequeace of feature values. The 
sequence of featote values is vtodow-mean filtered with a filter window lengfti of P ftames 
using the following fimotion! 

5 By using the filter fimction, the problem of noise and distortions is reduced. Due to varying 
signal conditions or encoding conditions the feature sequences can be distorted in xnultiple 
ways. Distortions could lead to a missed or a felse identification of a video sequence. 

Step 24 reduces the number of averages by using sub-sampling. Because a 
sequence of feature values is window mean filtered it could be sub-sampled without losing 

10 significant infbimation. Sub-sampling every F/2 period has the advantage that the total 

mnnber of datapointa inUie signature decreases with afectorF/2 and thus makes it possible 
to compare more signatures simultaneously, r is the sub-sampling rate, the defeiilt value is 
m assuming even F. K is the number of samples in the sub-sampled filtered sequeace. Kls a 
natiaral number that is rounded down if L-F+1 is not an integer multiple of r. 

Subsignatore (i» Ic) is the sub-sampled and filtered sequence of feature values in contentOc) in 
the filter window at timeCk) for feature Ij: 

20 subsIg»aturea,Jt)»[filt«0-.fc,0 filterC/,fc,2r) - Glt&r(J>k,JS>)] 

3tep 22 and 24 may bei^eated several thnes to create multiple sub-signatures for multiple 
features. Step 26 aeatea the first signatocensing the sub^goatures created in step 24. A 



25 



signature oonsiste of M sids-agnatures: 

8ignature(k)-[ sHl»signataw»"a*) •* Pubsie?iatare'(K,t) ] 



tfoder general conditions, the proposed signature can be generated veay efficiently during 
online operations. Every N frames a new signaUnreOw) of received or stored content canbe 
30 made. The first time, a complete signatoreM must be made. However, after that, a new 
slgnatmeOw) can easily be created usmg the N new frames, subsignature (j^knew^kow) equals 
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subsigtiature Q^ww) if N is a multiple of the sub-sanapling rate Content (Iw) comprises 
coaitent (kbw) andtimeOkn<»w)= time(kbu)+N. 

la step 82 shown in Fig. 6, PeatureSeq 0. Km, ^la) creates an updated 
S sequence of feature values fiom a feature ]^ in multiple frames in an updated sequeoce of 
frames: 

uewIteatareSeqO', 

= [feature(content(A:),time(A:)-iV+l,y) ... feature(conteat(it),time(A).;)] 
= §eBatiiwSeqC/,/r^)j^„ ... iBataxeStqXj.k,^)^ n«wFeatiWfeSeqaA«:,.)J 

10 Klter (j, koM,p) is the updated filter flmction for a feature Ij in multiple ftamcs in flie 
updated sequence of ftames: 



-S'^a^^yeSeqafc^.*^)^^,, otherwise 



15 pater G A>id,p) is pro-calculated. If N is an exact multiple of the sub-sampUng rate r, then 
2>=isr/t and subsignatore G JsnswlCoid), see step 84^ is Uie updated sub-saa:g)led filtered 
sequence. Subsi£nataireG,lCou)i^£H[^tiS^ ... 

20 

Step 6 shown in Fig.1 detemriniag similarity between the first and the second signatiro may 
b© subdivided into six st^ in a fevorable embodiment, see Kg3. In the favorable 
embodiment, pub-signatures are not compared as a whole but small sliding window 
sequences, caHed context windows, are compared instead. Using context windows solves the 
25 problem of shifls in timing between two aimilar or even eqtial snb-signatures, "Ehes© shifts 
can oeour because only every NT frames a signature is compared. Using context windows also 
solves the problem of local shifts in the sequence due to missing or ins^ed frames. Alihough 
conxparing fli6Fouriwi»ower-specti»of the sub-signatwesmay also solve ihis problem, since 
flie power specbum is invariant to shifts^ difEferences at the borders of the sub-signocores 
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could result in differences iu the power-spectra. Pur&Brmorei conoputational eftorta of this 

solution migto be mucli higher. 

Step 42 creates cotdext windows fiff the first and ibe second signatures created 
in steps 4 and 6 shown in Fig.1 . Context windows are created for each value in each sob- 
signature in both signatures and comprise multiple values from a sub-signatuce around a 
position in the sttWgnature. The Matrix of context windows for a subsignature(j,ki); 
sttb8ignatureC/»*i)j suIislgnatureC/.fci)^' 



subagnaftUfeC/.^iWti — wbsIgnfliarBC/.Ai)*, 



Step 44 calculates the correlation between each context window in a first sub^gnatore and 
10 eadii context window in a second sub-signature. The calculation comprises creating 
nonnalized context windows and calculating contextCk)irQ,ki.k2.Pi.p2)! 



std(cw''afc,)j 

[NotANumberCSlaJSOF"', std(cw^a*i)i.J=0 



coittc»tCoirC/,Jfci,*2,i't.Pa)«| W-\ ' 8td(ncw'"a*2;l»2))'*0 

NaN, olJiwvy»se 

15 Theproposedsinnlaiitymeasureis based on correlation. Correlation can always be 

consistently scaled between -I andl independem of &e mean and variance of ttosignatpres. 

Consequently correlation is also gf orogbustto-distortions-their^rinstaafle^oM^ 

Error, Context correlation is undefined if one of the window sequences is constant. Although 
another measure couldbe defined in case that one of the context window standard deviations 
is 2eto, Ibis will m^e me overall signature similarity measure inconsistent. Thus, e&ctively 
only the non-constant parts are compared which has the disadvantage that the comparison is 
less strict Incceashjg the content window width can increase the number of non-constaut 
partsi this however increases tiie computational load. Step 44 is repeated for ea^ai first sub- 
signature and each second sub-signature created for the same feature. 



20 
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Step 46 calculates a coefficieafi of coirelaticai contextSimG^Jculsz.p) between a 
context window at portion p in the first sub-signature aad multiple context windows in the 
second sub-signatvire. The final context window similarity at position p in subsignatureQ^i) 
with the context window at a corresponding position p in subsignatureCj^a) is defined as the 
5 best context ooirelation with the context window at neighborhood positions p-Ln to p+ of 
subsignature (jjfca). Ln is the neighborhood radius. QQskulsa^p) Is a set of positions fiom 
subsignature OJkj), flae positions being in the neighborhood of position p ftom subsignature 

10 QU>ki,k,,p)^y:{raiix{p^L„^}r'-,witi{p+L„,K~^ 



14:4. 



15 Step 46 is repeated fi>r each first sub-signature and each second sub-signatuie created fiw tihe 
same feature. 

Step 48 calculates a coefacient of ooirelation subSigSimQA^ between a 
^xst subsignature (j, ki) and a second subsignature 

subSigSimC/.;^../^,) = \\PuXm ,^:r^^'*-**'^^' ^O-,*..fc.)^0 

NaN, i'O".iti.Ag)«0 



As shown above the complete sub^gnatur© simiUiityis defined by the avar^e context 
similarities that are defined. If all context windows are constant tbs sub-sigaatqie similarity is 
not defined. Finally the complete signature similaa^ is defined as tihc average of defined 
25 sub-signature similarities. Step 48 is repeated for each first sub-signature and eadi second 
sab-signature created for the sanae feature. 

Step 50 calculates a coefifident of coirelation signatnieSiroOci,k2) between the 
first and the second signature. 
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1^ 



1+1 



) ..... _ 



' ^ NaN, JUA»^^^ 

The signature simfladty is seeded audh liiat its range is ftom zero to one. althou^ this ia not 
necessaiy. Note that the in extreme atttattonsHie sigoature moaarity can he undefined if one 

5 or both of the signatures are completely constanL 

Step 52 compares the coefedent a tibreshold. When the coefficiaxt is 
higher than the threshold, the first and the second signatore and hence the fiwt and second 
content item, e.g. andioMdeo sequences, can be identified as h^ equal. When the 
sigaatures are too simple. i.e. not specific enongh. a good threshold will not ejdsL Ibece ate 

10 multiple signstwre generation parameters lhat can be varied to increase the spedfloity of the 
signatures, Ideatification quality oonldbe further improved by generating nmlt^le agnatores 
for an audio/video sequence at multiple time instances, for instance^ attnne(k). time(k)+G, 
time(kH2G, etc. In order to i«3entify1fae sequence a largepaoentege of ihe generated 
signatures should be positively identified. This in^ves the robustness and qoaUty of tiie 

15 idoxttfication mechanism. 

Weights may be used in step 46 to calculate the coefficient of cojrelafian 
conteKtSim0^ul«2.P) at position p in the first sub-signature and multiple context windows hi 
the second sub-signature of ihe second signature, a weight being larger if a context window 
in the second sub-signatore is near tibie coiresponding positionp and smaller if fhp second 

20 element is remote ftom the corresponding position p. ConteKtSinti(|,ki,kM>) is redefined to 
incorporate a wei^ w(p,q)t 



max (>*'(i>.g)cont8*tCotrC/,*i.fe,i'.«)) Qi)A>h^P')*^ 



25 Thewd^fimctionw(p,q)is aWockfimctlonif aEcontextwindowsinthesecondsuV 
signature that are in Ihe ndghborhood of the coiresponding position p have equal weight 
With Uup weight ftnotioa the original formulation as previously defined is preserved: 
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The wd^t fimotion w(p,q) ia a triangular fiiaotion if a weight is used such that coctext 
windows further ftom coiresponding position p are less important: 



w 



0, otherwise 



10 



15 



is the triangle base length. 

Similarity can be evaluated efBoiently during online operations. Every N 
frames a new signature of received or stored content is made and conxpared witti mijltiple 
reference signatures. For eacb. reference sub-signature(3,ki) a context correlation matrix: 
CC(iMM) is maintained containing tiie context correlation of each context window of sub- 
signatureCj Jtl) witb all context windows in sub-signature(jjjk2). 



• • • 



A contsct sinailadly matdx is cabulateid xtsiag neigjibodiood-wdisbling malxix. W: 



20 W = 



w(l,l) ••• w(ff-B^+U) 



The cwtecict dmildzity matrix: 



25 
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Thematrix ms(A) operation finds ite maximum pwcoliami of A. AUNaNelem 
are(Kscardedftomtliemaximmnop€JalioJuIfaUelemeatsofaoolwmM^ 
maximal value for that column is NaN . The * . * » operator is the element-wis e matrix 
multiplication operator. SubSigSjmC'^uka) and sigaatareSimOci^ oanbe caJcuJated Wig 
5 the context similarity matrix. 

Because an updated signatnreOcanew) wliere time(fcjn«w) niinus time(k2oi^ 
equals H only contains Z (=N/r) new values at the end of the sub-signatures only Z new 
nomialized context windows are calculated. For tiie Z new context windows in sub- 
sigQatHreG,k2„«v) the context correlation with liie (K-W+l) context windows of sub- 
10 signatureCki) is calculated. These correlation values are used to update the context 

correlation matrix CJCf^XM"^ CCQ, Ici. W)- The Z new normaUzed context windows in 
snlmgDiaCure (j>^0'> 

Xhe new context cQirdLatioiL matrix: 

nexurCCU^i.^a)"-" 

15 CCC/»fri,&2^.*2»«5 . 

It is assumed that any linear operation with aKaN results in aNaN. Thus, if 
one or both of ^e normalized context windows is constant fee resulting contort coirdation is 
NaN, Using the iip^ed context correlation matdces all the new similatities can be 
20 calculated. 

T:M^tectconicTfevtee^-of^Figr4-^omprises-^-intei^ 



wiUi a storage means 66 storing a first signature of a first content item, the first content item 
comprising a first sequence of frames. The device 62 further compdaes a receiver 68 able to 
receive a signal comprising a second content item, the second content item cwr^iising a 
25 second sequence of frames. The device 62 also comprises a control unit 70 able to n^e the 
interfeoe 64 to retrieve tiie first signature from the storage means 66. able to czeate a second 
signature for ihe second content item, and able to detemune similarity between the first 
signWe and the second signature. The control unit 70 is able to create a first sub-signature 
from I3ie jQrst sigpature. ihe first sub^signature comprising a first sequence of averages of 



co.t^u^.cADiac LH'H^ rnxuxi-a oxr nu -rax cr^wms:? 

PHNL020800EP^(p ^ 021 26. 08. 2002 14:4, 



15 26.08.2002 
values of a feature in multiple flames in the first sequence of frames. The first sub-sigaatuie 
may be extracted Scorn, the first signature or if the first signatare comprises raw data, e.g. a 
sequence of feature values, the first sub-signature may be calculated in the same way as the 
second sub-signature. The first signature may also need to be processed in other waj^ to 
5 create ttie first sub-signature. The control unit 70 is able to create a second sub-signature fcr 
the second signature by averaging values of the feature in multiple ftames in the second 
segueaice offices. The control unit 70 is able to detemiine similanlty between the first and 
the second sub-signature. The control unit 70 is able to determine similarity between ttie first 
and the second signature in dependency of Uje similari^ between flie first and ttia S6c<md 
10 sub-signature. The storage means 66 may be comprised m 13ie device 62 or may be an 

external device. The storage means 66 may for example conqwise a hard disk or an optioal 
storage medium. The receiver 68 may receive a signal using cable 76. The receiver 68 may 
fijT example receive signals flom a cable operator or fixnn a satellite dish. 

The control unit 70 maybe able to determine similarity between the first and 
15 the second signature by calculating a coefficient of ooirelation between the first and the 

second signature and comparing the coefiBcient wilii a Ihteshold. If tiie second contetit iiam is 
comprised in a third content item and the first and «k» second signature are similar, the 
control unit 70 may be able to urge a fijrther stomge means 72 to store ttie fliird content item 
without the secondcontent item. The contiol unit 70 mayhe able to urge a further storage 
20 means 72 to storelhe second contentitemifthefirstandthe second signature are simi^ 
The flirther storage means 72 tnay be coinprised in Has d9vie& 62 or may be an external 
device. The fiarther storage means 72 may for etxanqile con^xise a hard disk or an optical 
storage medium. The fbrther storage means 72 and fii© storage means 66 may be physically 
or logically different parts of the same hardware. The control unit 70 may be able to use a 
25 further interf^ 78 to retrieve data fi»mfljefinther storage m«ans 72. The inte^ 

the fintfaer intatfiice 78 maybe pbysjoally or logically different parts of the same hardware. 

The control unit 70 maybe able to generate an alert if the first and the second 
signature are shmlar. The alert maybe displayed using a display 74. The alert may also be 
audible. If ilie device 62 is a Digital TV, ihe display 74may be comprised in the device 62. If 
30 the device 62 is a Personal Video Reconte, the display 74 maybe an external device. The 
display 74 may for example be a CaiT, a JjCD. or a Plasma display. The user may be 
responsible for initiating the creation of the first signature. He or she could press a * generate 
signature' button on a remote control of a PVR at the moment a geoerio intro of aprogcam is 
shown. Afl:er the button is pressed, Ote PVR could ask the us er what to do when the first 
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sign^e and the second signature siimlar. If tto Tjser wan^ 
he or she may be able to specify the idativereccading start time ai^ end to^ 
of chamels to scan. For instance -3 xnin. 00 sec to +30 inm 00 sec on ABC, CBS, and NBC. 
If a user wants to be alerted, he oi she may be able to specify a set of ohannela to scan. The 
5 user may also be able to indicate lhat an occoirenoe of a srcnilar signatore is to be stored in a 
database enabling a usei to jump to content or to sldp content during playback. 

The PVR may also be able to search for a second signature similar to fhd fiist 
signature in a coUeotion of stored content and play back Ihe second content item if the second 
signature is found. In this way auser oonld jump from the start of one stored episode to «io 
10 start of another stored episode of the same series. Anoiher way to jump is to have predefined 
signatures. Auser maybe able to select a specifio first signature ftom a list of signatures. 
With a button-press the user can jmnp to the next histance of an kitro. Instead of usmg a list, 
a small set of signatures could be programmed by the user on the remote oonttol. If a user 
always likes to watch a specific news show or a specific TV comedy he or she coidd program 
15 generic buttons on the remote control to link to these programs using the predefined 

signatures. If a user is playing back stored content and presses the generic button that links to 
the specific news show, the PVR wiU jump to a next identified intro of the specific news 
show. If the button is pressed again «ie PVR will jump again toanext identified intro. The 
first and die second signature may be compared 'W^ile the second content item is being stored 

20 in Ihecollectioa of stored content 

While the invention has been described in connection with preferred . , . 

embodhnents, it will be understood that modifications thereof within the principles outlined 
above will be evident to those skilled in the art, and thus the invention is not limited to the 
preffetted embodiments but is intended to encompass such modifications. The invention 

25 resides in each and evCTy novel characteristic feature and each and every combination of 
oharaoteristic features. Refferenoe numerals m the ciarais oo not imitth-^pttStBctivffwopB: 
Use of lh6 verb 'Ho comprise" and its conjugations does not exclude 1he presence of elements 
olher than those stated in the claims. Use of toe article "a" or "an" preceding an element does 
not ejwlude the presence of a plurality of such elements. 

3Q 'Means', as will be apparent to a person skilled hi the ait, are meant to inchide 

any hardware (such as separate or integrated circuits or eleotconic elements) or software 
(such as programs or parts of programs) which perform in operation or are designed to 
petfinm a ^edfied fim^tion, be it solely or in conjunction with other functions, be it in 
isolation or in oo-operadon with other elements. The invention canbe implemented by means 
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of hardware oomprising several distinct dements, and by means of asuitably programmed 
computer. Ih the device claim enumerating several means, several of these means can be 
embodied by one and the same item of hardware, 'Software' is to be imderstood to mean any 
software product stored on a computer-readable medium, such as a floppy disk, 
downloadable via a network, such as the Inteniet, or marketable in any other manner, 
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CLAIMS! 



1. A method of content Ideatifioatioji, comprising the step ofi 

sealing a fiist signature fer a fiist conteat item comprisiog a first sequence of firames, 
cljaraeteiized in Ihat 

fh© ptep of creating the first aignatnxe conq)rises creating a first sub-sigftature to comprise a 
5 first sequence of fiist averages, a first average bdng sttioken of vahjes of a feature in 
mutliple fiames In the first sequence of fiames. 

2. A method as claimed in daanil, characterized in that; 

arther comprised is the step of creatuig a second d.gnature for a second content item 
10 comprising a second sequence of fi'smes; 

tbs step of creating the second signatare comprises creating a second sub-signature to 

conqvise a second seqpience of second averages, a second average bemg stricken of values of 

the feature in multiple fiames in tiie second sequence of ftames; 

fijither comprisedis the step of delecminiDg similarity between the first 3nd the second 
IS signature; and 

title step of detramining similarity between the first and the second signature comprises 
detetmloing amilarity between the first and fiie second sub-signatore. 

3. Ainslhoda8ClaimedinoIaim2,charaoteri2edinthattiiestQ>ofdetermio^ 
20 sirailarily between the first and the second signatare conqoises calculating a coefficient of 

,^iAttftft hatweeo the first and the second signature and comparing the coemcient with a 
thr^hold. 

4. A method as claimed in claim 2, diaraoterized in &at the step of determining 
25 fiimflaritybfltweenthefitst and the second signatare conqmsescalcd 

ccttrdWbetween afitst sub-sequence at a position in the first sequence of averages and 
multiple second sub-sequences in the naghbothood of a corresponding position in the second 
sequence of averages. 
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5. A method as claimed in claim 4, characterized in fbat &e coefiSeieixt of 
conrelation betiveen the first snb-seguence and the multiple second sub-seqaences is 
calculated using weights, a weight being larger if a second sub-sequence is near the 
corresponding position and smaller if a second sub-sequence is remote firom the 

5 corresponding position. 

6. A method as claimed in claim 2, characterized in that the step of creating a 
signature comprises creating multiple sub-signatures and similarity between Ihe first and the 
second signature is determined using Ihe multiple sub-signatures. 

10 

7« A method as claimed in claim 2, characterized in that creating a sub-signature 

comprises reducing the number of averages. 

8- A method as claimed in claim 2, characterized in that if the second content 

1 5 item is comprised in a third content item and the first and the second signature are similar, a 
• lUrther step is comprised skipping the second content item in the third content item. 



^- A method as claimed in claim 2, characterized in that a furth^ st^ is 

conipiised identifying boundaries between a first segment and a second segment of a third 
20 content item and another step is comprised skipping the first segment in the third content 
item if the second content item comprises the first segment and the first and the second 
signature are similar. 



10. A mettiod as claimed in claim 2, characterized in that a further step is 

25 comprised recording the second content item if the first and the second signature are similar, 

11. A method as claimed in claim 2, characterised in that a fiirthef step is 
comprised generating an alert if the first and the second siguafxure are similar. 

30 12. An electronic device, comprising: 

an intedGace fbr int^J^oing with a storage means storing a first signature of a first contmt 
item> the first content item comprising a first sequence of frames; 

a receiver able to receive a signal comprising a second content item, the second content item 
comprising a second sequ^ce of firames; and 
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acontrol unit able to use the toterfeoe to retrieve the &st dgnatgre fiomttie storagemeana, 
able to createasecc«d signature fbr the second content item, and *letodet^ 
between the first signature and lihe second signature, 
oharactarized in that: 

5 uie Qontrol unit is able to create a first sub-signature ftom the first signature, the first sub- 
signature comprising a first sequaice of avwages of values of a feature in multiple frames in 
the first sequence of frames; 

tHe control unit is able to create a second anb^gnature for flie second signature by averaging 
values ofthe feature inmultiplefirames in the second sequence of fiamesi 
10 the control unit is able to determine similarity between the first and the second sub-signature; 

the control unit is able to determine similaiily between the first and Ihe second signature in 
dependency of the sinnlarity between the first and the second sub-signature. 

15 13. A device as clairned in claim 12, characterized in that the control unit is able 

to detennine similarity between ihe first and the second signature by calculating a coefficient 
of correlation between the first and the second signature and comparing th^ coefficient with a 
tixteshold. 

20 14. A device as claimed in claim 12, characterized in that if the second content 

item is comprised m a third content item and the first and the second signatoro are similar, the 
Qontrol nnit is able to urge a finiher Storage means to store the third content item Wifliont the 

second content item. 

25 15. A device as claimed in claim 12, characterized ux that the control unit is able 

to urge a fiirther storage means to -^fe-lhe-second-conteaHtemiif^e-firsfr^n^e^ eeend 

signature are similar. 

1 6. A device as claimed in claim 12, characterized in that the control unit is able 
30 to generate an alert if the first and the second signature are similar. 

17. Software enabling upon its execution apiogrammable device to fimotlon as an 
electronic device, cgmprising a function for creating a signature for a content item 
compriBing a sequence of fi^mes, the fimction comprising creating a sub-signature to 
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comprise a sequence of averages, an average being ^ckea of values of a featare in multiple 
frames in the sequence of frames* 



18. Software as claimed in claim 17, characterized by fbrther comprising a 

5 ftnction for detormirdng similaiity between two signatures by calculating a coefficient of 
correlation between the two signatures and comparing the coefiSoient with a threshold 

19. Software as claimed in claim 17, charactacizsed by being stored on an 
infoimatiozi carrier. 
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ABSTRACT: 



10 



The toethod of centent identifioatiott consists of creating a signature to 
comprise on© or mote sub-signatmes. A sub-signaftxre is created by averaging values of a 
feature inmmtiple frames of a content item (24). The electronics device (62) is able to 
letrieve a first signature of a first content item fiom a storage means (66) and is able to 
receive a second content item nsing a receiver (68). The device has a control rnrit (70) able to 
create one or more sob-dgnatgres by averaging values of one or more features in miUtiple 
frames of thje second content item andusingthe one or more sub-signatures to create a 
second signature. The control unit (70) is also able to detemiine similarity between the two 
sigoatures by detenmning similarity of sub-signatures fer a similar feature. The software is 
able to createasigoatuie for aconteatitemby averaging values of afea»ic8 in m 

fianies in a sequence of ftames in the contart itran. 
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